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Abstract 

Planning, implementing, and assessing a service-learning project 
can be a complex task because service-learning projects often 
involve multiple constituencies and aim to meet both the needs 
of service providers and community partners. In this article, 
Stufflebeams Context, Input, Process, and Product (CIPP) evalu¬ 
ation model is recommended as a framework to systematically 
guide the conception, design, implementation, and assessment 
of service-learning projects, and provide feedback and judgment 
of the projects effectiveness for continuous improvement. This 
article (1) explores the CIPP evaluation model’s theoretical roots 
and applications, (2) delineates its four components, (3) analyzes 
each component’s role in a service-learning project’s success, 
and (4) discusses how the model effectively addresses Service- 
Learning Standards for Quality Practice. This article illustrates 
the application and evaluation of the model in a teacher-educa¬ 
tion service-learning tutoring project. 


Introduction and Review of Literature 

J ervice-learning in educational settings involves the integra¬ 
tion of community service into the academic curriculum 
(Koliba, Campbell, & Shapiro, 2006). Service providers achieve 
curricular goals while providing services that meet a community’s 
needs (Zhang, Griffith, et al, 2009; Zhang, Zeller, et al., 2008). A suc¬ 
cessful service-learning project thus requires that a faculty member 
identify the needs of service providers and community partners, 
design a project that can effectively address both needs, and imple¬ 
ment the project in a manner that generates the desired outcome. 
Each of these steps is vital to the success of the service-learning 
project; each requires, therefore, careful monitoring to ensure 
its effective execution. Moreover, service-learning projects often 
involve multiple stakeholders. They can generate unanticipated 
outcomes as well as intended outcomes. Although assessments 
aiming at one or several impacts, and focusing on a single stage of 
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a service-learning project, can be informative in answering specific 
questions on the value of service-learning, they cannot systemati¬ 
cally guide the planning and implementation of service-learning 
projects. To date, no evaluation model appears to be widely adopted 
by faculty members to guide service-learning projects. The authors, 
therefore, posit in this article that Stufflebeam’s (2003) Context, 
Input, Process, and Product (CIPP) model can serve as a guiding 
framework for service-learning projects. 

Approaches for Evaluating Projects 

There appear to be some 26 approaches often employed to 
evaluate projects. These 26 may be grouped into five categories: 
pseudoevaluations, quasi-evaluation studies, improvement- and 
accountability-oriented evaluation, social agenda and advocacy, and 
eclectic evaluation. The first category, pseudoevaluations, includes 
five approaches that are often motivated by political objectives: 
public relations-inspired studies, politically controlled studies, 
pandering evaluations, evaluation by pretext, and empowerment 
under the guise of evaluation. The other 21 approaches are typically 
used legitimately to judge projects. The quasi-evaluations include 
14 approaches that either focus on answering one or several ques¬ 
tions or use a single methodological approach: objectives-based 
studies; accountability, particularly payment-by-results studies; 
success case method; objective testing programs; outcome evalua¬ 
tion as value-added assessment; performance testing; experimental 
studies; management information systems; benefit-cost analysis; 
clarification hearing; case study evaluations; criticism and connois- 
seurship; program theory-based evaluation; and mixed-methods 
studies. The improvement/accountability category is oriented 
toward determining the merit and worth of the project or entity 
being evaluated, and encompasses three approaches: decision- and 
accountability-oriented studies, consumer-oriented studies, and 
accreditation and certification. The social agenda/advocacy cat¬ 
egory dedicates evaluation efforts to pursuing social justice and 
includes three approaches: responsive evaluation or client-centered 
studies, constructivist evaluation, and deliberation democratic 
evaluation. Finally, the eclectic evaluation category includes the 
utilization-focused evaluation approach, and draws selectively 
from all available evaluation concepts and methods to serve the 
needs of a particular user group (Stufflebeam & Shinkfield, 2007). 

When compared with professional standards for project evalu¬ 
ation, and after being rated by their utility, feasibility, propriety, 
and accuracy, the best approach that has surfaced is the Context, 
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Input, Process, and Product evaluation model. The CIPP evaluation 
model belongs in the improvement/accountability category, and is 
one of the most widely applied evaluation models. Unlike more 
traditional evaluation approaches such as the Tylerian Evaluation 
Rationale (Tyler, 1942), which is an objectives-based approach in the 
quasi-evaluation category mainly concerned with the final retroac¬ 
tive evaluation of whether a set of objectives has been met, the CIPP 
evaluation model is designed to systematically guide both evalua¬ 
tors and stakeholders in posing relevant questions and conducting 
assessments at the beginning of a project (context and input evalu¬ 
ation), while it is in progress (input and process evaluation), and 
at its end (product evaluation). A survey by American Society for 
Training and Development members found that the CIPP model 
was preferred over other evaluation models (Galvin, 1983). 

What the CIPP Evaluation Model Can Do 

Specifically, the context evaluation component of the Context, 
Input, Process, and Product evaluation model can help identify 
service providers’ learning needs and the community’s needs. The 
input evaluation component can then help prescribe a responsive 
project that can best address the identified needs. Next, the process 
evaluation component monitors the project process and potential 
procedural barriers, and identifies needs for project adjustments. 
Finally, the product evaluation component measures, interprets, 
and judges project outcomes and interprets their merit, worth, sig¬ 
nificance, and probity. 

Planning, Implementing, and Assessing Service- 
Learning Projects: A Multifaceted Task in Need 
of a Guiding Framework 

The challenge of carrying out service-learning projects lies in 
the complexity resulting from multiple project objectives and mul¬ 
tiple participating groups (e.g., faculty and community members, 
and students). The challenges are intensified by the lack of a reliable 
evaluation model that systematically guides the service-learning 
projects (Zhang, Zeller, et al., 2008; Zhang, Griffith, etal., 2009). 

The need for rigorous and authentic assessment of service¬ 
learning outcomes has been increasingly recognized, and the 
many challenges in assessing service-learning have been enumer¬ 
ated (Butin, 2003; Gelmon, 2000a; Holland, 2001). Service-learning 
is a complex approach to teaching and learning; it needs and 
deserves approaches to assessment, evaluation, and reporting 
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that are capable of capturing that complexity (Eyler & Giles, 1999; 

Karayan & Gathercoal, 2005; Mabry, 
1998; Moore, 1999; Pritchard, 2002; 
Steinke & Buresh, 2002; Troppe, 
1995). 

A number of effective ser¬ 
vice-learning project assess¬ 
ments focusing on specific 
aspects of service-learning out¬ 
comes have been conducted 
(Bringle & Kremer, 1993; Hamm, 
Dowell, & Houck, 1998). These 
assessments represent the per¬ 
formance testing approach in the 
quasi-evaluation category. For 
example, Furco ( 2002 ) developed 
the Self-Assessment Rubric for Institutionalizing Service-Learning 
in Higher Education, a tool that enables colleges and universities to 
measure the degree to which service-learning is part of the institu¬ 
tion’s culture. Marchel (2004) discussed evaluating reflection and 
sociocultural awareness, and Peterson (2004) discussed assessing 
performance in problem-based service-learning projects. The pre¬ 
dominantly used data collection mechanisms include survey meth¬ 
odology (Kezar, 2002) and reflection (Ash & Clayton, 2004). Portfolios 
have also been recommended (Banta, 1999). 

A few studies have examined the complexity of service-learning 
by focusing on the various groups of people involved, a method 
resembling the client-centered evaluation approach. A case study 
model of assessment was developed at Portland State University, 
to measure the impact of service-learning on four constituencies: 
students, faculty, community, and the institution (Driscoll, Holland, 
Gelmon, & Kerrigan, 1996). Subsequent work by Driscoll, Gelmon, 
et al. (1998) has provided an assessment model for service-learning 
projects specifically in education that focuses on the four constitu¬ 
encies of service-learning. The model provides both quantitative 
and qualitative measures at three levels of assessment: diagnostic, 
formative, and summative. Additionally, Gelmon, Holland, et al. 
( 2001 ) have offered useful principles, techniques, and tools for 
assessing service-learning and civic engagement. 

Holland ( 2001 ) suggested a more comprehensive evaluation 
model for assessing service-learning based on a goal-variable- 
indicator-method design, which can be best characterized as an 
objectives-based evaluation approach. Its strength is its attention to 


“Service-learning is 
a complex approach 
to teaching and 
learning; it needs and 
deserves approaches to 
assessment, evaluation, 
and reporting that are 
capable of capturing 
that complexity.” 
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the complex dynamics behind service-learning—the collaborative 
work of students and faculty members (within their institutional 
context) with their community partners. Holland’s work serves as 
the first step toward providing an evaluation model for assessing 
service-learning. However, the lack of a sense of sequence and 
intertwined nature may limit its usefulness. Currently, it appears 
that no specific evaluation model has emerged as an easy-to-use, 
systematic guide to a service-learning project’s planning and imple¬ 
mentation (e.g., Baker-Boosmara, Guevara, & Balfour, 2006; Bordelon, 
2006; Borges & Hartung, 2007). 

Stufflebeam’s Context, Input, Process, and 
Product Evaluation Model: An Improvement 
Accountability Approach 

Stufflebeam’s Context, Input, Process, and Product evalua¬ 
tion model is “a comprehensive framework for conducting forma¬ 
tive and summative evaluations of projects, personnel, products, 
organizations, and evaluation systems” (Stujflebeam & Shinkfield, 
2007, p. 325). The model originated in the late 1960s to provide 
greater accountability for the U.S. inner-city school district reform 
project. It was to address the limitations of traditional evaluation 
approaches (Stujflebeam, 1971 ). The CIPP evaluation model “is con¬ 
figured especially to enable and guide comprehensive, systematic 
examination of social and educational projects that occur in the 
dynamic, septic conditions of the real world . . .” (Stujflebeam & 
Shinkfield, 2007, p. 351 ). Over the years, the model has been refined 
(Alkin, 2004) and used by a wide range of disciplines (Stujflebeam & 
Shinkfield, 2007). 

In education settings, the CIPP evaluation model has been 
used to evaluate numerous educational projects and entities 
(Zhang, Griffith, et al, 2009; Zhang, Zeller, et al, 2008). For example, 
Felix (1979) adopted the model to evaluate and improve instruc¬ 
tion in Cincinnati, Ohio, school systems. Nicholson (1989) recom¬ 
mended the CIPP evaluation model to evaluate reading instruc¬ 
tion. Matthews and Hudson ( 2001 ) developed guidelines for the 
evaluation of parent training projects within the framework of the 
CIPP evaluation model. A faculty development project designed 
to support the teaching and evaluation of professionalism of med¬ 
ical students and residents was examined using the CIPP evalu¬ 
ation model (Steinert, Cruess, Cruess, & Snell, 2005). The model was 
used to construct Taiwan’s national educational indicator systems 
(Chien, Lee, & Cheng, 2007). The model also served as the evaluation 
model for Osokoya and Adekunle (2007) to assess the trainability 
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of enrollees in the Leventis Foundation (Nigeria) Agricultural 
Schools’ projects. Moreover, Combs, Gibson, et al. (2008) derived 
a course assessment and enhancement model based on the CIPP 
evaluation model because of its flexibility in providing formative 
and summative results. 

Over the years, exemplary applications of the model within 
education occurred in numerous evaluations by Bill Webster of 
the Dallas, Texas, Independent School District; Howard Merriman 
of the Columbus, Ohio, school district; Gary Wegenke and his 
evaluators of the Des Moines, Iowa, school district; Jerry Baker 
of the Saginaw, Michigan, school district; Jerry Walker of The 
Ohio State University National Center for Research on Vocational 
Education; Bob Randall of the Southwest Regional Educational 
Research Laboratory; Carl Candoli and his evaluators of the 
Lansing, Michigan, school district; Stufflebeam and colleagues of 
the Evaluation Centers (first at The Ohio State University and sub¬ 
sequently at Western Michigan University); and others. Many of 
the reports from these applications of CIPP were archived in ERIC 
centers, and some appeared in dissertations (D. L. Stufflebeam, per¬ 
sonal communication, April 16, 2010). 

The CIPP evaluation model emphasizes “learning-by-doing” 
to identify corrections for problematic project features. It is thus 
uniquely suited for evaluating emergent projects in a dynamic 
social context (Alkin, 2004). As Stufflebeam has pointed out, the most 
fundamental tenet of the model is “not to prove, but to improve” 
(Stufflebeam & Shinkfield, 2007, p. 331). The proactive application of 
the model can facilitate decision making and quality assurance, 
and its retrospective use allows the faculty member to continually 
reframe and “sum up the project’s merit, worth, probity, and sig¬ 
nificance” (Stufflebeam & Shinkfield, 2007, p. 329). 

The link between the unique features of the CIPP evalua¬ 
tion model and the need for a systematic comprehensive guiding 
framework for service-learning projects is strong. Stufflebeam and 
Shinkfield illustrate this link with this observation: 

The Context, Input, Process, and Product evaluation 
model has a strong orientation to service and the prin¬ 
ciples of a free society. It calls for evaluators and clients 
to identify and involve rightful beneficiaries, clarify 
their needs for service, obtain information of use in 
designing responsive projects and other services, assess 
and help guide effective implementation of service, and 
ultimately assess the services’ merit, worth, significance, 



Using the Context, Input, Process, and Product Evaluation Model as a Comprehensive Framework 63 


and probity. The thrust of CIPP evaluations is to provide 
sound information that will help service providers regu¬ 
larly assess and improve services and make effective and 
efficient use of resources, time, and technology in order 
to serve the well-being and targeted needs of rightful 
beneficiaries appropriately and equitably (2007, p. 330) 

In summary, the authors believe that the model can help guide 
service-learning project needs assessment and planning, monitor 
the process of implementation, and provide feedback and judg¬ 
ment of the project’s effectiveness for continuous improvement. 

Understanding the Model 

The four components of the Context, Input, Process, and 
Product evaluation model are useful in guiding the stages of a 
service-learning project. This section delineates the four compo¬ 
nents of the model and demonstrates each component’s role as 
applied to a project. The authors discuss how the model addresses 
the Service-Learning Standards for Quality Practice (National Youth 
Leadership Council, 2008). Finally, the authors describe the appli¬ 
cation of the model using a teacher-education service-learning 
tutoring program. 

The Four Components 

All four components of Stufflebeam’s CIPP evaluation model 
play important and necessary roles in the planning, implemen¬ 
tation, and assessment of a project. According to Stufflebeam 
(2003), the objective of context evaluation is to assess the overall 
environmental readiness of the project, examine whether existing 
goals and priorities are attuned to needs, and assess whether pro¬ 
posed objectives are sufficiently responsive to assessed needs. The 
purpose of an input evaluation is to help prescribe a program by 
which to make needed changes. During input evaluation, experts, 
evaluators, and stakeholders identify or create potentially relevant 
approaches. Then they assess the potential approaches and help 
formulate a responsive plan. Process evaluation affords opportuni¬ 
ties to assess periodically the extent to which the project is being 
carried out appropriately and effectively. Product evaluation identi¬ 
fies and assesses project outcomes, both intended and unintended. 
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Component I. 

Context evaluation is often referred to as needs assessment. It 
asks, “What needs to be done?” and helps assess problems, assets, 
and opportunities within a defined community and environmental 
context (Stufflebeam & Shinkfield, 2007). According to the authors, 
the objective of context evaluation is to define the relevant context, 
identify the target population and assess its needs, identify oppor¬ 
tunities for addressing the needs, diagnose problems underlying 
the needs, and judge whether project goals are sufficiently respon¬ 
sive to the assessed needs. The methods for the context evaluation 
include system analyses, surveys, document reviews, secondary 
data analyses, hearings, interviews, diagnostic tests, and the Delphi 
technique (Dalkey & Helmer, 1963). 

The context evaluation component addresses the goal identi¬ 
fication stage of a service-learning project. An effective service¬ 
learning project starts with identifying the needs of service pro¬ 
viders (students) and the needs of the community. Many pitfalls 
are associated with needs assessments. Most can be attributed to 
the failure of adequate identification and articulation, in advance, 
of crucial indicators (e.g., purpose, audience, resources, and dis¬ 
semination strategies). Application of the context evaluation com¬ 
ponent of the CIPP evaluation model could potentially prevent 
these pitfalls. 

Component II. 

Input evaluation helps prescribe a project to address the 

identified needs. It asks, “How 
should it be done?” and identi¬ 
fies procedural designs and edu¬ 
cational strategies that will most 
likely achieve the desired results. 
Consequently, its main orienta¬ 
tion is to identify and assess cur¬ 
rent system capabilities, to search 
out and critically examine poten¬ 
tially relevant approaches, and to 
recommend alternative project 
strategies. The result of the 
input evaluation step is a project 
designed to meet the identified 
needs. The success of a service¬ 
learning project requires a good project plan that, if implemented 


“The success of a 
service-learning 
project requires a good 
project plan that, if 
implemented correctly, 
will benefit both service 
providers (students) 
and service recipients 
(community members).” 
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correctly, will benefit both service providers (students) and ser¬ 
vice recipients (community members). Methods used to execute 
an input evaluation include inventorying and analyzing available 
human and material resources, proposed budgets and schedules, 
and recommended solution strategies and procedural designs. 
Key input evaluation criteria include a proposed plan’s relevance, 
feasibility, superiority to other approaches, cost, and projected 
cost-effectiveness. Literature searches, visits to exemplary proj¬ 
ects, employment of advocate teams, and pilot trials are all appro¬ 
priate tools to identify and assess alternative project approaches. 
Once a project plan is developed, it can be evaluated (using tech¬ 
niques such as cost analyses, logic models, Program Evaluation and 
Review Techniques [PERT], and various scales) according to the 
criteria that were identified in the input evaluation step (Stufflebeam 
& Shinkfield, 2007). 

Component III. 

Process evaluation monitors the project implementation pro¬ 
cess. It asks, “Is it being done?” and provides an ongoing check 
on the project’s implementation process. Important objectives of 
process evaluation include documenting the process and providing 
feedback regarding (a) the extent to which the planned activities 
are carried out and (b) whether adjustments or revisions of the 
plan are necessary. An additional purpose of process evaluation 
is to assess the extent to which participants accept and carry out 
their roles. 

Process evaluation methods include monitoring the project’s 
procedural barriers and unanticipated defects, identifying needed 
in-process project adjustments, obtaining additional informa¬ 
tion for corrective programmatic changes, documenting the 
project implementation process, and regularly interacting with 
and observing the activities of project participants (Stufflebeam 
& Shinkfield, 2007). Process evaluation techniques include on-site 
observation, participant interviews, rating scales, questionnaires, 
records analysis, photographic records, case studies of partici¬ 
pants, focus groups, self-reflection sessions with staff members, 
and tracking of expenditures. 

Process evaluation can be especially valuable for service¬ 
learning projects because (a) it provides information to make 
on-site adjustments to the projects, and (b) it fosters the develop¬ 
ment of relationships between the evaluators (in this case, the two 
task force members in research and evaluation methodology) and 
the clients/stakeholders that are based on a growing collaborative 
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understanding and professional skill competencies, which can pro¬ 
mote the projects long-term sustainability. 

Component IV. 

Product evaluation identifies and assesses project outcomes. It 
asks, “Did the project succeed?” and is similar to outcome evalua¬ 
tion. The purpose of a product evaluation is to measure, interpret, 
and judge a project’s outcomes by assessing their merit, worth, sig¬ 
nificance, and probity. Its main purpose is to ascertain the extent 
to which the needs of all the participants were met. 

Stufflebeam and Shinkfield (2007) suggest that a combina¬ 
tion of techniques should be used to assess a comprehensive set 
of outcomes. Doing so helps cross-check the various findings. A 
wide range of techniques are applicable in product evaluations, 
and includes logs and diaries of outcomes, interviews of beneficia¬ 
ries and other stakeholders, case studies, hearings, focus groups, 
document/records retrieval and analysis, analysis of photographic 
records, achievement tests, rating scales, trend analysis of longitu¬ 
dinal data, longitudinal or cross-sectional cohort comparisons, and 
comparison of project costs and outcomes. 

Providing feedback is of high importance during all phases of 
the project, including its conclusion. Stufflebeam and Shinkfield 
(2007) suggest the employment of stakeholder review panels and 
regularly structured feedback workshops. They stress that the 
communication component of the evaluation process is absolutely 
essential to assure that evaluation findings are appropriately used. 
Success in this part of the evaluation requires the meaningful and 
appropriate involvement of at least a representative sample of stake¬ 
holders throughout the entire evaluation process. 

Product evaluation used in service-learning projects can serve 
at least three important purposes. First, it provides summative 
information that can be used to judge the merits and impacts of 
the service-learning project. Second, it provides formative infor¬ 
mation that can be used to make adjustment and improvement to 
the project for future implementation. Third, it offers insights on 
the project’s sustainability and transportability, that is, whether the 
project can be sustained long-term, and whether its methods can 
be transferred to different settings. 
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Table I. Applying the Context, Input, Process, and Product Evaluation 
Model to the Service-Learning Standards for Quality Practice 
Standards for Quality Practice Context, Input, Process, and Product 

Framework 


Service-learning actively engages participants in mean¬ 
ingful and personally relevant service activites. 

Context evaluation: Identify participants’ 
needs. 

Input evaluation: Design project that is 
engaging and targets partcipants’ needs. 

Service-learning is intentionally used as an instruc¬ 
tional strategy to meet learning goals and/or content 
standards. 

Context evaluation: Identify learning 
goals. 

Input evaluation: Design project as an 
effective instructional strategy to meet 
learning goals. 

Service-learning incorporates multiple challenging 
reflection activities that are ongoing and that prompt 
deep thinking and analysis about oneself and one’s 
relationship to society. 

Input evaluation: Design project that 
includes multiple challenging reflection 
activities. 

Process evaluation:Assess reflection 
activities through reflective journals, 
focus group interviews, and surveys on 
self-perceptions. 

Service-learning promotes understanding of diversity 
and mutual respect among all participants. 

Input evaluation: Design project that will 
promote understanding of diversity and 
mutual respect among all participants. 

Process evaluation: Formatively and 
summatively assess whether the project 
promoted understanding of diversity and 
mutual respect among all participants. 

Service-learning provides youth with a strong voice in 
planning, implementing, and evaluating service-learning 
experiences with guidance from adults. 

Context, input, process, and product 
evaluation: Involve participants in plan¬ 
ning, implementing, and evaluating 
service-learning project. 

Service-learning partnerships are collaborative, mutually 
beneficial, and address community needs. 

Context evaluation: Identify participants’ 
and community needs. 

Input evaluation: Design project that is 
mutually beneficial and allows partici¬ 
pants to work collaboratively to address 
community needs. 

Service-learning engages participants in an ongoing 
process to assess the quality of implementation and 
progress toward meeting specified goals, and uses 
results for improvement and sustainability. 

Process and product evaluation: Engage 
participants in an ongoing process to 
assess the quality of implementation and 
progress toward meeting specified goals, 
and use results for improvement and 
sustainability. 

Service-learning has sufficient duration and intensity 
to address community needs and meet specified 
outcomes. 

Context evaluation: Identify community 
needs, and specify intended outcomes. 

Input evaluation: Design project with 
sufficient duration and intensity. 

Process and product evaluation:Assess 
whether community needs and specified 
outcomes are met. 

The Context, Input, Process, and Product 
Evaluation Model’s Linkage to Service-Learning 
Standards for Quality Practice 

In 2008, the National Youth Leadership Council devised the 
K-12 Service-Learning Standards for Quality Practice (National 
Youth Leadership Council, 2008). The standards, which were vetted 
through a series of “reactor panels” convened nationwide by 
the National Youth Leadership Council and RMC Research 
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Corporation, serve as a yardstick for judging the quality of K-12 
service-learning practices. Table 1 outlines how the CIPP evalua¬ 
tion model can serve as an organizing framework for systematic 
evaluation of service-learning projects to help meet the Standards 
for Quality Practice. 

It is evident that the CIPP evaluation model has unique fea¬ 
tures that can help effectively address the K-12 Service-Learning 
Standards for Quality Practice. These unique features include 
context evaluation, ongoing process evaluation, and the model’s 
emphasis on engaging participants in the evaluation process. The 
CIPP evaluation model can help provide youth with a strong voice 
in planning, implementing, and evaluating service-learning expe¬ 
riences; engage participants in an ongoing process to assess the 
quality of implementation and progress toward meeting specified 
goals; and use evaluation results for improvement and sustainability. 

Applying the Model to a Teacher-Education 
Service-Learning Tutoring Project 

During spring semester 2008, a service-learning tutoring 
project was initiated to address the learning needs of pre-service 
teachers in the Elementary Education project at a public research 
university in the southeastern United States, as well as the needs 
of at-risk readers in the local school system. Twenty-six pre-ser¬ 
vice teachers taking a course in diagnostic/prescriptive teaching 
of reading completed a service-learning assignment by tutoring 26 
Response-to-Intervention students (RTI at-risk readers) in kinder¬ 
garten, first grade, and second grade. University Internal Review 
Board (IRB) human subjects approval was secured for this project. 
The CIPP evaluation model was used to guide the conception, 
design, implementation, and assessment of the tutoring project. 
The authors believe that its use led to the project’s achieving the 
desired outcomes. The CIPP components were implemented as 
shown in Table 2 and in the narrative below. 



Using the Context, Input, Process, and Product Evaluation Model as a Comprehensive Framework 69 


Table 2. Using the Context, Input, Process, and Product Evaluation Model 
to Guide a Service-Learning Tutoring Project 


Context, Input, Process, and Product 
Evaluation Model Components 

Methods Used in the Service-Learning 

Tutoring Program 

Component 1: Context evaluation 


Assessed the setting for the intended 
service. 

Identify the needs, and the assets and opportunities 
for addressing the needs. 


Interviewed school principal, teachers, and 
reading specialists. 

Reviewed school records. 



Identified at-risk readers and their needs. 



Administered diagnostic tests given to 
at-risk readers. 

Conducted initial quantitative assessment 
of at-risk readers. 



Conducted pre-service teacher focus 
group interviews. 



Conducted initial quantitative assessments 
of pre-service teachers. 

Component II: Input evaluation 


Reviewed relevant literature. 

Prescribe a project to meet the identified needs, 
and identify and assess project strategies and pro¬ 
cedural designs. 


Interviewed school principal, teachers, and 
reading specialists. 

Consulted university reading faculty mem¬ 
bers and other experts. 



Viewed exemplary projects. 



Consulted Learn and Serve America. 



Formed advocate teams. 



Service-learning task force members met 
biweekly. 



Conducted pre-service teacher focus 
group interviews. 

Component III: Process evaluation 


Identified what activities should be 
monitored. 

Monitor project’s process and potential procedural 
barriers, and identify needs for project adjustments. 


Received biweekly update from service¬ 
learning task force. 

Observed service-learning activities. 



Kept a log of the activities. 



Interviewed at-risk readers. 



Interviewed pre-service teachers. 



Interviewed school principal, teachers, and 
reading specialists. 

Reviewed pre-service teachers’ 
self-reflections. 



Reviewed students’ work samples. 



Conducted debriefing with pre-service 
teachers. 

Component IV: Product evaluation 


Conducted post-project quantiative 
assessments of pre-service teachers. 

Measure, interpret, and judge project outcomes, 
and interpret their merit, worth, signficance and 
probity. 


Conducted post-project focus group 
interview of pre-service teachers. 

Conducted post-project quantitative 
assessment of at-risk readers. 



Administered at-risk readers’ survey. 



Interviewed or surveyed other stake¬ 
holders, including faculty instructor, 
principal, teacher, reading specialist, and 
parents of at-risk readers. 
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Assessing Community Needs Using Context 
Evaluation 

The community involved in the teacher-education service¬ 
learning project included the university initiating the project and 
the local school system. The university was assessed as to its needs 
for and capability of carrying out the service-learning project. The 
university’s college of education offered both undergraduate and 
graduate projects of study in teacher preparation. All of its projects 
incorporated internship opportunities or capstone projects in col¬ 
laboration with K-12 school professionals and other community 
partners. Service-learning became part of the culture at this univer¬ 
sity. The creation of a new position, the vice chancellor for service¬ 
learning, generated a surge of interest in service-learning projects 
and service-learning opportunities on and off campus. The uni¬ 
versity had a well-established infrastructure for service-learning 
research activities. University curricula included well-integrated 
projects that enable students to spend a semester or more in a con¬ 
nected series of courses linked to service-learning projects outside 
the university. 

The needs within the university. 

Following a review of curriculum vitae and discussions among 
faculty members, a service-learning faculty task force was created 
for this project; it consisted of one university administrator, eight 
expert content area faculty members, and two faculty members in 
research and evaluation methodology who served as evaluators of 
the project. The service-learning task force examined the univer¬ 
sity’s mission, curriculum, professional teaching standards, class 
experiences, literature, and feedback from school systems, and then 
identified the need within the college to work toward improving 
teacher retention. It was further ascertained that pre-exposure to 
the school environment decreases teacher attrition. That is, if pre¬ 
service teachers have a better understanding of the teaching pro¬ 
fession and the reality of working with diverse student populations 
through hands-on practice, they are more likely to acquire needed 
professional pedagogical skills and to stay in the teaching field once 
they enter it. 

The needs within the local school system. 

To identify community needs within the local school system, 
the task force communicated with and interviewed adjunct faculty 
members who were also teachers in the school system. Assistance 
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to elementary-level at-risk readers topped the needs list. Following 
these meetings, the task force asked, “What kind of service-learning 
project will best meet both the needs of the elementary-level at-risk 
readers and the need of our pre-service teachers to gain firsthand 
experience with diverse student populations?” and “Which school 
has the greatest needs?” The task force looked for a school situ¬ 
ation that provided the best fit between the pre-service teachers’ 
needs and the needs of children. Once a potential site was iden¬ 
tified, the researchers met with the principal and discussed their 
proposal. Students in the Response-to-Intervention process (RTI 
at-risk readers) were selected to be the service-learning recipi¬ 
ents because they were working below grade level in reading and 
writing. They were considered at-risk readers and writers, but were 
not receiving special education services. This target population of 
at-risk readers needed, but had not been receiving, individual assis¬ 
tance in reading. 

The service site was an elementary school that is very repre¬ 
sentative of a typical elementary school in the county; the school’s 
racial and socioeconomic balance was the result of countywide 
district school equity policies. The principal was very receptive to 
the proposed service-learning project. The elementary teachers 
involved were positive about the potential benefits of the project 
for their students and were pleased to work with the pre-service 
teachers and the university faculty. 

Assessment of pre-service teachers’ readiness. 

Initial assessments of the pre-service teachers were conducted 
in January, before the service-learning intervention, in five ses¬ 
sions of focus group interviews to explore their initial attitudes 
and dispositions about this project. The interviews revealed that 
the pre-service teachers were equipped with the knowledge and 
skills needed to provide the service. More importantly, they 
expressed curiosity and a strong desire to participate in this project. 
Quantitative instruments were also used prior to project imple¬ 
mentation to assess the pre-service teachers’ initial level regarding 
the following constructs: community service self-efficacy, moti¬ 
vations regarding volunteer activity, self-esteem, and confidence 
in making a clinically significant contribution to the community 
through service. These quantitative research instruments included 
the following: 

• The Self-esteem Scale (Rosenberg, 1965) 
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• The Community Service Self-efficacy Scale (Reeb, 
Katsuyama, Summon, & Yoder, 1998) 

• The Volunteer Functions Inventory (Clary, et al, 1998) 

• The Personal Social Values Scale (Mabry, 1998) 

Assessment of at-risk readers’ needs. 

Similarly, additional context evaluation-related assessments 
were completed on the initial status of elementary school students’ 
self-esteem, and their steps toward independence and academic 
achievement in reading and oral and written language skills. The 
pre-service teachers administered literacy assessments, including 
running records (Clay, 1993), the Qualitative Reading Inventory-4 
(Leslie & Caldwell, 2006), the Elementary Reading Attitude Survey 
(McKenna & Rear, 1990), and the Burke Reading Interview (Burke, 
1980). Based upon these assessments, the pre-service teachers 
designed and taught lessons that targeted the students’ needs while 
building upon their strengths. 

Formulating Plans Using Input Evaluation 

Input evaluation was completed in order to prescribe a sound 
service-learning project. Meetings were conducted with university 
reading faculty members, reading specialists in the school, and 
potential collaborating classroom teachers to discuss what kind 
of service-learning projects would best meet the students’ needs. 
Based on information gathered from the input evaluation process, 
a tutoring project that joined pre-service teachers in a reading 
methods course with a selected cohort of RTI at-risk readers was 
prescribed. Each week during the 15-week semester course, the 
pre-service teachers were assigned to spend 3 hours and 30 min¬ 
utes on preparation for their tutoring experience and 1 hour and 
30 minutes providing direct tutoring service to an identified RTI 
at-risk reader. 

Next, an extensive literature review on best practices for 
working with at-risk readers was conducted as part of the initi¬ 
ated service-learning project. The task force members spoke with 
faculty members in the area of reading education in this university 
and other universities. They also discussed the plan with and sought 
feedback from reading specialists, in the targeted school. Finally, 
the task force watched videos of exemplary service-learning proj¬ 
ects, visited leading service-learning websites, and discussed ele¬ 
ments that would be important to include in this project. 
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As an important part of the input evaluation, expert input 
was sought to judge the feasibility of the service-learning tutoring 
project before its implementation, and adjustments were made 
to improve the project. Contributing experts included mem¬ 
bers of Learn and Serve America (the funding agency of the 
project), several nationally recognized experts in service-learning, 
the university’s vice chancellor for service-learning, and the 
Department of Curriculum and Instruction chairperson. Based 
on the input received, the task force held face-to-face discussions 
as well as Delphi studies, a structured communication technique 
(Dalkey & Helmer, 1963), to refine the plan. The improved plans were 
then shared with the principal and cooperating teachers for their 
input. 

Monitoring Progress Using Process Evaluation 

To assess the process, the task force members held biweekly 
meetings to give updates on the project’s implementation. They 
also shared positive stories and discussed any potential problems 
that needed to be addressed. The task force members held regular 
discussions with the collaborating teachers, the principal, and the 
reading specialists. 

The university faculty member who taught the reading course 
and her graduate assistant observed the service-learning activi¬ 
ties regularly and kept a detailed log. Feedback regarding needed 
adjustment was gathered and acted upon. For example, the needs 
for guidance on administering an assessment and modifying 
instruction for English learners were identified and promptly 
addressed. Assessment guidance was provided by reviewing with 
the pre-service teachers step-by-step instructions on how to prop¬ 
erly administer the assessment. Instruction for English learners was 
modified by providing slower paced instruction that was accom¬ 
panied with explanation of unusual words. The faculty member 
also held weekly in-class debriefings with the pre-service teachers 
on the service-learning project. The pre-service teachers verbally 
reported on the activities, progress, and challenges in the service¬ 
learning project, and the instructor led class discussions to address 
issues that arose. 

Pre-service teachers’ self-reflection served as an important 
window into the operation of the project and its impact on them. 
Self-reflection has been recognized as an essential link between 
community experience and academic learning (Ash & Clayton, 2004; 
Felten, Gilchrist, & Darby, 2006). Reflection can also serve as a mirror 
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that makes pre-service teachers’ inner changes visible to the faculty 
instructor and the project evaluators. Following each of the tutoring 
sessions, the pre-service teachers spent 15-20 minutes reflecting on 
what they had gained from the tutoring session, and what they had 
contributed to the students and the cooperating teacher. 

To monitor the ongoing impacts of the tutoring sessions on 
the RTI at-risk readers, formal and informal academic assess¬ 
ments, structured observations, and curriculum-based measure¬ 
ments were employed during the project. Formal assessments are 
more “standardized” assessments: Tests are uniform in procedures 
for being administered, amount of time allowed for completion, 
and method of grading. Informal assessments are more flexible 
in their usage. They are designed by teachers specifically for their 
classrooms/students and/or for a certain lesson or topic. Informal 
assessments can also be things that are part of the daily classroom 
routine. Structured observation is an observation in which the 
observer completes a questionnaire or counts the number of times 
an activity occurs. The curriculum-based measure is a method of 
monitoring student educational progress through direct assess¬ 
ment of academic skills (Marston, Mirkin, &Deno, 1984). When using 
curriculum-based measures, the instructor gives the student brief 
samples, or “probes,” made up of academic material taken from 
the child’s school curriculum. These curriculum-based measure 
probes are timed and may last from 1 to 5 minutes. The child’s 
performance on a curriculum-based measure probe is scored for 
speed, or fluency, and for accuracy of performance. The results are 
then charted to offer the instructor a visual record of a targeted 
child’s rate of academic progress. The cooperating teachers as well 
as the faculty instructor regularly observed the service-learning 
tutoring activities and provided oral and written feedback to the 
pre-service teachers. 

Assessing Impact Using Product Evaluation 

The product evaluation was centered on two overarching ques¬ 
tions: (a) Did the service-learning experience meet the pre-service 
teachers’ learning needs? (b) What impacts did the service-learning 
reading skills tutoring project have on the at-risk readers? The 
impact of the program on pre-service teachers was assessed using 
various data, including pre-service teachers’ own reflections; direct 
quantitative assessments using survey research scales; focus group 
interviews of pre-service teachers; performance on the assignments 
for the reading methods course in which the pre-service teachers 
were enrolled; faculty observations of tutoring sessions; and input, 
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such as interview responses and other verbal feedback received 
informally, from university faculty advisors, the elementary school 
principal, participating teachers, reading specialists, and RTI at- 
risk learners. 

Assessing pre-service teachers’ learning 

Reflection. 

The pre-service teachers’ reflective journal entries, which 
were part of the process evaluation component of the Context, 
Input, Process, and Product mode, were collected and entered 
into QSR International’s NVivo 8 qualitative data analysis soft¬ 
ware (NVivo Qualitative Data Analysis Software, 2008) and ana¬ 
lyzed qualitatively by three experienced researchers. These three 
researchers included the two task force members in research 
methodology and another task force member whose area of 
expertise is reading education. The researchers are not involved 
in the project’s implementation process. Following the work of 
MacQueen, McLellan, Kay, and Milstein (1998); Fernald and Duclos 
(2005); and Fonteyn Vettese, Lancaster, and Bauer-Wu (2008), the 
three researchers adopted a codebook structure and the iterative 
process of discussing each code until agreement was reached. 
They then worked independently and collaboratively, gave each 
code a definition, set inclusion and exclusion criteria, and iden¬ 
tified sample text references from the transcripts. Each reflective 
journal entry was independently coded by the three members of 
the team. Disagreements were resolved through discussion at our 
weekly meetings so that the codes were further refined (Fonteyn, et 
al, 2008). The authors believed that this process enhanced inter¬ 
coder consistency. Wherever possible, team members attempted 
to use participants’ own words “to guide the construction of codes 
and their definitions for in-depth analysis,” a process referred to 
by MacQueen, et al. (1998) as emic or nonstructural coding (p. 33). 
Findings from the reflection logs revealed an increase over the 
course of the semester in the pre-service teachers’ understanding of 
the reading process and, more importantly, of their roles in helping 
students develop effective reading processes. 

Focus group interviews. 

Five sessions of focus group interviews were conducted before 
the service-learning intervention (as part of the context evaluation) 
and again after the service-learning intervention (as part of the 
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product evaluation). These interviews with the pre-service teachers 
explored whether the service-learning tutoring experience had 
changed their level of confidence in “making a difference,” any of 
their personal social values in regard to service-learning and com¬ 
munity service, or their attitudes and dispositions toward working 
with students from diverse backgrounds. These interviews were 
video-recorded directly to DVD, transcribed, and analyzed using 
NVivo 8 (NVivo Qualitative Data Analysis Software, 2008). The results 
revealed that the service-learning group had positive changes over 
the course of the semester in terms of their levels of confidence in 
“making a difference,” their personal social values in regard to ser¬ 
vice-learning and community service, and their attitudes and dis¬ 
positions toward working with students from diverse backgrounds. 

Quantitative assessments of affective learning 
using standardized research scales. 

Quantitative instruments were used before the service¬ 
learning project (as part of the context evaluation) and after the 
service-learning project (as part of the product evaluation) to 
assess changes in pre-service teachers regarding their community 
service self-efficacy, motivations for volunteer activity, self-esteem, 
and confidence in making a considerable contribution to a com¬ 
munity through service. The following data collection instru¬ 
ments were used before and after the project: the Self-esteem Scale 
(Rosenberg, 1965), the Community Service Self-efficacy Scale (Reeb, 
et ai, 1998), the Volunteer Functions Inventory (Clary, et al, 1998), 
and the Personal Social Values Scale (Mabry, 1998). The pre-service 
teachers’ responses on these research scales were statistically ana¬ 
lyzed, and positive changes were found after the service-learning 
experience regarding self-esteem, community service self-efficacy, 
motivation to volunteer, and personal social values as they relate 
to community service. 

Other assessments of pre-service teachers’ 
learning. 

Using process and product evaluation, pre-service teachers’ 
academic performances were monitored throughout the service¬ 
learning experience. The university faculty instructor regularly 
conducted observations of the tutoring sessions. Samples of pre¬ 
service teachers’ coursework, faculty observation field notes, cur¬ 
riculum-based measures, and reflective journals were collected and 
assessed by the university faculty instructor to explore the pre-ser- 
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vice teachers’ understandings and mastery of the reading process 


“These pre-service 
teachers demonstrated 
more diligence and 
better quality in 
their academic work 
than the pre-service 
teachers... who 
were not involved in 
service-learning.” 


“responsible,” “caring,” “skilled,” “patient,” and “the children love 
them.” 

Assessing Impact on At-Risk Readers 

The effect of the service-learning project on the at-risk 
readers’ self-esteem; steps toward independence; and academic 
achievement in reading, and oral and written language skills, was 
assessed through formal and informal academic assessment, struc¬ 
tured observations, curriculum-based measures, and students’ 
reflective journals. The assessment measures were employed during 
the project (during process evaluation) and at the end of the project 
(during product evaluation). Elementary students’ perceptions 
of themselves as readers, oral communicators, and writers were 
assessed prior to their participation in the project (context evalu¬ 
ation) and after the conclusion of the project (product evaluation) 
by using the Elementary Reading Attitude Scale (McKenna & Rear, 
1990). A survey of the participating elementary school teachers 
regarding their assessment of the impact of the project on the at- 
risk readers was also administered. These results indicated that 
the at-risk readers benefited from the project through increases in 
reading ability, self-esteem, and self-perception of themselves as 
readers, as well as improved attitudes toward reading. 

The CIPP evaluation model served as a guiding framework and 
systematically guided the conception, design, implementation, and 
assessment of this service-learning project in teacher-education. 
First, the Context Evaluation component identified the pre-service 


and reading instruction. These 
pre-service teachers demon¬ 
strated more diligence and better 
quality in their academic work 
than the pre-service teachers in 
another section of the course the 
same instructor was teaching, 
who were not involved in service¬ 
learning. Community members 
who worked closely with them 
(e.g., the principal, cooperating 
teachers, and reading special¬ 
ists) were interviewed regarding 
their perceptions of these pre¬ 
service teachers. Their responses 
included descriptions such as 
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teachers’ need to gain firsthand experience working with students 
from diverse backgrounds and the elementary school RTI at-risk 
readers’ need for individualized assistance in reading. Next, Input 
Evaluation incorporated input from experts, practitioners, and 
various stakeholders, and prescribed an effective tutoring project. 
Then Process Evaluation helped monitor the project implementa¬ 
tion process and provided ongoing feedback for needed adjust¬ 
ments. Finally, Product Evaluation assessed the service-learning 
project’s impacts and provided feedback and judgment of the proj¬ 
ect’s effectiveness. 


Conclusion 

University-based service-learning projects involve multiple 
stakeholders and aim to meet the needs of service providers and 
community partners. Their complex and dynamic nature calls for 
an evaluation model that can operationalize the process and pro¬ 
vide step-by-step systematic guidance. Effectiveness is essential 
to the continued viability and growth of service-learning projects 
throughout the United States. 

The issue of multiple goals and multiple constituencies is a 
major challenge in evaluating service-learning. Without a guiding 
evaluation model that is well-aligned with the unique features of a 
service-learning project, assessing the project maybe challenging. 
Service-learning projects are too often subject to suboptimal assess¬ 
ments despite the collection of massive amounts of data, because 
researchers lack both the knowledge of key elements to assess, 
and access to a reliable evaluation model to organize the data and 
present the findings to various stakeholders in meaningful ways. 

Without effective evaluation, service providers cannot make 
their projects and services better (Stufflebeam and Shinkfield, 2007). 
For example, service providers cannot be sure their goals are worthy 
unless they validate the goals’ consistencies with sound values and 
a structured responsiveness to service recipients’ assessed needs. 
Service providers cannot plan effectively and invest their time 
and resources wisely if they do not identify and assess options. 
It may be difficult to sustain university-community partnerships 
if the leaders of service-learning projects cannot show that they 
have responsibly carried out the project plan, produced beneficial 
results, and met the needs of the community partner. 

Though not a new model, the CIPP evaluation model intro¬ 
duced in this article can be especially useful for guiding the plan¬ 
ning, implementation, and assessment of service-learning projects. 
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The model employs multiple methods, has been tested in a wide 
range of contexts, has evolved and strengthened over time, and 
is supported by theoretical and pragmatic literature (Stufflebeam 
& Shinkfield, 2007). The model not only assesses the impact of a 
service-learning activity, but also helps to identify community 
needs by working with the community to identify needs and goals 
to be addressed and to formulate a project targeted to best meet 
those identified community needs, monitor project implementa¬ 
tion, evaluate project outcomes, and provide recommendations for 
project improvement. 

Without the guidance of the Context, Input, Process, and 
Product Evaluation Model, oversight or failure can easily occur in 
any part of the process, which could seriously hinder the service¬ 
learning projects operation and diminish its effectiveness. For 
example, without the model’s regulation, the needs may not be as 
carefully identified, the match between the needs of participants 
may not be as meticulously ensured, problems in the implementa¬ 
tion process may not be identified and corrected in a timely manner, 
and necessary multiple assessment methods may not be designed 
into the assessment. Each of these elements plays an important role 
in the service-learning projects success. 

It is particularly important to note that because the Context, 
Input, Process, and Product evaluation model is a social systems 
approach to evaluation, all participants in a service-learning project 
help design the activities that they agree will meet the articulated 
needs of both the service providers (university members) and ser¬ 
vice recipients (community members). Shared decision making 
is essential, because “sustained, consequential involvement posi¬ 
tions them to contribute information and valuable insights and 
inclines them to study, accept, value, and act on evaluation reports” 
(Stufflebeam & Shinkfield, 2007, p. 330). This social systems approach 
fosters an understanding and connection among service providers, 
community partners, and other stakeholders and can effectively 
promote long-term sustainability of a service-learning project. 

From conceptualizing a service-learning project to insti¬ 
tutionalizing it in the curriculum requires informed planning, 
guided implementation, and evidence of impact. A snapshot type 
of assessment from a single lens using mono-method construc¬ 
tions is unlikely to provide the kind of comprehensive and mul¬ 
tifaceted data needed by educational policymakers. Stufflebeam’s 
CIPP evaluation model has the potential to guide faculty members 
using service-learning as a teaching tool to systematically gather 
assessment data at each stage of a service-learning project, so that 
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they can make informed judgments to sustain or improve the 

project. 
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